Image processing apparatus and method, and non-transitory computer-readable storage medium

ABSTRACT

An image processing apparatus of the present invention for processing image data obtained by an image capturing apparatus comprises one or more processors and/or circuitry which functions as: an output unit that outputs the image data obtained by the image capturing apparatus and maximum luminance information which is information for indicating a maximum luminance of the image data to an external device; an acquisition unit that acquires a maximum luminance that the image capturing apparatus is capable of handling; and a control unit that controls the output unit to output the maximum luminance acquired by the acquisition unit as the maximum luminance information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2018/021859, filed Jun. 7, 2018, which claims the benefit ofJapanese Patent Application No. 2017-126515, filed Jun. 28, 2017, bothof which are hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing apparatus andmethod, and non-transitory computer-readable storage medium.

Background Art

Heretofore, there are technologies for transmitting a video signal to adisplay device such as a television from a video signal source thatholds video content. In recent years, technologies for transmittingcharacteristics of video content (e.g., information on the maximumluminance of content, etc.) that are included in video content to adisplay device as metadata, together with video information from thevideo signal source, when transmitting video content, have becomeprevalent.

By using such technologies, it becomes possible for the display deviceto perform image processing suitable for the performance of the displaydevice and the characteristics of the video content. Also, it becomespossible for the side transmitting the video signal to transmit theirintention as to how they want the video signal to be displayed on thedisplay device, through the metadata that is appended.

As such technologies, there exist technologies for transmitting placesof interest in video content, luminance information on objects and thelike as metadata (refer to PTL1).

On the other hand, there are current digital cameras that connect to atelevision using a cable that conforms to standards such as HighDefinition Multimedia Interface (HDMI) (registered trademark), and havea function of displaying images recorded on recording media in thedigital camera on the display of a television.

Here, in the case where luminance information is appended as describedabove, it is conceivable, when information on luminance differs fromimage to image, to transmit metadata that includes information on thehighest luminance that differs from image to image to the television. Insuch cases, the metadata will differ from image to image, and thetelevision may perform image processing that differs from image toimage.

As an example of image processing that depends on the maximum luminanceinformation attached as metadata, in the case where the maximumluminance information is less than or equal to the displayable luminanceof the display device, display is performed at the luminance per thevideo signal. On the other hand, in the case where the maximum luminanceinformation exceeds the displayable luminance of the display device, auniform gain is applied to the input signal, and display is performed ata reduced luminance overall. With such image processing, in the casewhere a plurality of images of the same main subject are continuouslydisplayed, for example, the following problem arises when images whosemaximum luminance within the image is higher than and lower than thedisplayable luminance of the display device are displayed together, evenif the luminance of the main subject is the same. That is, the luminanceof the main subject that is displayed fluctuates up and down, adverselyaffecting the viewing experience.

The present invention has been made in view of the above problem, and anobject thereof is to be able to, in the case of displaying a pluralityof images, suppress any change in luminance when a subject having thesame luminance throughout the plurality of images is represented on adisplay device.

CITATION LIST Patent Literature

PTL1: Japanese Patent Laid-Open No. 2016-34125

SUMMARY OF THE INVENTION

In order to achieve the above object, an image processing apparatus ofthe present invention for processing image data obtained by an imagecapturing apparatus has one or more processors and/or circuitry whichfunctions as: an output unit that outputs the image data obtained by theimage capturing apparatus and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; an acquisition unit that acquire a maximum luminancethat the image capturing apparatus is capable of handling; and a controlunit that controls the output unit to output the maximum luminanceacquired by the acquisition unit as the maximum luminance information.

Further, according to the present invention, provided is an imageprocessing apparatus capable of processing a plurality of image datahaving a plurality of luminance ranges, comprising one or moreprocessors and/or circuitry which functions as: an output unit thatoutputs image data to be output and maximum luminance information whichis information for indicating a maximum luminance of the image data toan external device; a sorting unit that sorts a plurality of image dataobtained by image capture; and a control unit that controls the outputunit to output, as the maximum luminance information, the maximumluminance of a plurality of image data sorted into a predeterminedgroup, rather than the maximum luminance of the image data to be output,in a case of outputting, with the output unit, one piece of image datasorted into the predetermined group.

Furthermore, according to the present invention, provided is an imageprocessing method for processing image data obtained by an imagecapturing apparatus, comprising: outputting the image data obtained bythe image capturing apparatus and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; acquiring a maximum luminance that the image capturingapparatus is capable of handling; and controlling to output the acquiredmaximum luminance as the maximum luminance information.

Further, according to the present invention, provided is an imageprocessing method capable of processing a plurality of image data havinga plurality of luminance ranges, comprising: outputting image data to beoutput and maximum luminance information which is information forindicating a maximum luminance of the image data to an external device;sorting a plurality of image data obtained by image capture; andcontrolling to output, as the maximum luminance information, the maximumluminance of a plurality of image data sorted into a predeterminedgroup, rather than the maximum luminance of the image data to be output,in a case of outputting, in the output step, one piece of image datasorted into the predetermined group.

Further, according to the present invention, provided is anon-transitory computer-readable storage medium storing a program forcausing a computer to function as units of the image processingapparatus for processing image data obtained by an image capturingapparatus, comprising: an output unit that outputs the image dataobtained by the image capturing apparatus and maximum luminanceinformation which is information for indicating a maximum luminance ofthe image data to an external device; an acquisition unit that acquire amaximum luminance that the image capturing apparatus is capable ofhandling; and a control unit that controls the output unit to output themaximum luminance acquired by the acquisition unit as the maximumluminance information.

Further, according to the present invention, provided is anon-transitory computer-readable storage medium storing a program forcausing a computer to function as units of the image processingapparatus capable of processing a plurality of image data having aplurality of luminance ranges, comprising: an output unit that outputsimage data to be output and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; a sorting unit that sorts a plurality of image dataobtained by image capture; and a control unit that controls the outputunit to output, as the maximum luminance information, the maximumluminance of a plurality of image data sorted into a predeterminedgroup, rather than the maximum luminance of the image data to be output,in a case of outputting, with the output unit, one piece of image datasorted into the predetermined group.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the description, serve to explain the principles of theinvention.

FIG. 1 is a conceptual diagram showing a system in embodiments of thepresent invention.

FIG. 2 is a block diagram showing a configuration of processing modulesof a digital camera in a first embodiment.

FIG. 3 is a block diagram showing a configuration of processing modulesof a digital camera in second and fourth embodiments.

FIG. 4 is a diagram showing information that is included in imageinformation in the second embodiment.

FIG. 5 is a block diagram showing a configuration of processing modulesof a digital camera in a third embodiment.

FIG. 6A is a diagram for describing an example of grouping processing inthe third embodiment.

FIG. 6B is a diagram for describing an example of grouping processing inthe third embodiment.

FIG. 6C is a diagram for describing an example of grouping processing inthe third embodiment.

FIG. 7 is a diagram showing an example of grouping information in thethird embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, modes for carrying out the present invention will bedescribed in detail with reference to the accompanying drawings. Notethat, in the present embodiment, description is given using a digitalcamera as an example of an information processing apparatus, but thepresent invention is not limited to a digital camera, and is applicableto devices for processing information such as mobile phones, personalcomputers (laptops, desktops, tablets, etc.), and game machines.

FIG. 1 is a conceptual diagram showing a system in which a digitalcamera of the present embodiment is used. A digital camera 1 isconnected to a display 2 with a cable 3 (or wirelessly).

In the digital camera 1, shot images are stored as image files. Here,image files of the present embodiment refer to RAW files consisting ofinformation obtained by recording optical signals acquired by a sensorof the digital camera and shooting time information, and JPEG filesconsisting of image information converted into JPEG and shooting timeinformation.

In the system shown in FIG. 1, the digital camera 1 receives aninstruction 40 (hereinafter, reproduction instruction) for reproducingand displaying image files stored in the digital camera 1 on the display2 from a user 4. The digital camera 1, upon receiving the reproductioninstruction 40, performs display of images stored in the digital camera1 using the display 2, by transmitting a transmission signal 10consisting of image information and metadata generated from informationof the image files to the display 2.

First Embodiment

FIG. 2 is a block diagram representing the configuration of processingmodules in the digital camera 1 of the first embodiment, and, forexample, some or all the processing is constituted using a CPU, a memoryand the like (not shown) that are included in the digital camera 1. Inthe first embodiment, a user instruction acquisition unit 101, areproduction display control unit 102, a luminance informationacquisition unit 103, a dynamic range information holding unit 104, ametadata generation unit 105, a transmission signal generation unit 106,an image file holding unit 107 and a transmission unit 108 are includedas processing modules. Note that, in FIG. 2, processing modules that areunnecessary in describing the present embodiment are omitted.

The user instruction acquisition unit 101 consists of a user interfacesuch as buttons, a touch panel and the like arranged on the digitalcamera 1, for example, for receiving instructions from the user and aprocessing unit thereof, and receives the reproduction instruction 40from the user 4. The reproduction instruction 40 includes an instructionas to which image file to display among the plurality of image files andhow the image file is to be displayed. The user instruction acquisitionunit 101, upon receiving the reproduction instruction 40 from the user,transmits a reproduction control start instruction 1010 to thereproduction display control unit 102.

The reproduction display control unit 102, upon receiving thereproduction control start instruction 1010, transmits a luminanceinformation acquisition instruction 1020 to the luminance informationacquisition unit 103, and transmits a metadata generation instruction1021 to the metadata generation unit 105. Also, a transmission signalgeneration instruction 1022 is transmitted to the transmission signalgeneration unit 106. Here, information that is transmitted from thereproduction display control unit 102 to the processing modules includesinformation on the image to be displayed and how the image is to bedisplayed.

The luminance information acquisition unit 103, upon receiving theluminance information acquisition instruction 1020, acquires dynamicrange information 1040 showing the luminance range that is handled bythe digital camera 1 from the dynamic range information holding unit 104as luminance information. The dynamic range of the digital camera 1generally changes due to factors such as shooting settings, and thedynamic range information 1040 in the first embodiment is the maximumdynamic range capable of being handled by that model of digital camera1. As an example, here, description will be given assuming thatinformation showing 1000% is held in the dynamic range informationholding unit 104, as the maximum dynamic range capable of being handledby the digital camera 1. Note that the maximum value of the dynamicranges set at the shooting times of a plurality of images to bedisplayed may also be used.

Next, the luminance information acquisition unit 103 converts theacquired dynamic range information 1040 into luminance information 1030.In the first embodiment, the value of the dynamic range described in thedynamic range information 1040 is replaced by luminance, which is takenas luminance information. For example, 1000% is replaced by 1000 nits.

The metadata generation unit 105 receives the metadata generationinstruction 1021 and the luminance information 1030, and generatesmetadata. Here, in the first embodiment, description is given showingdata that includes information on maximum luminance as metadata, but thetype of metadata is not limited thereto. For example, information oncolor gamut, color temperature, minimum luminance and the like may beappended as metadata. Also, for example, information on MaxCLL (MaximumContent Light Level) or MaxFALL (Maximum Frame Average Light Level)supported by HDMI may be appended.

The metadata generation unit 105 converts the acquired luminanceinformation 1030 into a format of metadata (format defined as a metadataformat), and outputs the converted luminance information as metadata1050.

The transmission signal generation unit 106 receives the transmissionsignal generation instruction 1022 and the metadata 1050, and generatesa signal for transmitting to the display 2 connected to the digitalcamera 1. Specifically, the transmission signal generation unit 106,upon receiving the transmission signal generation instruction 1022,acquires image information 1070 required in creation of the signal to betransmitted from the image file holding unit 107. The image file holdingunit 107 holds image data having various dynamic ranges shot with thedigital camera 1 as image files. The transmission signal generation unit106 then converts the acquired image information 1070 into a formatsuitable for signal transmission.

In the present embodiment, image information is converted into a formatthat conforms to EOTF (Electro Optical Transfer Function) defined inSMPTE ST 2084 as a format suitable for transmission. Here, the signaldefined in SMPTE ST 2084 is a signal representing absolute luminance.However, generally, information of an image file shot with a digitalcamera is often held at a relative brightness, rather than being held ina data format representing absolute luminance. In view of this, in thepresent embodiment, allocation of a specific relative value (e.g., 18%)within each image file to a specific absolute luminance (e.g., 18 nits),is decided, and the format is converted so as to achieve that allocationwhatever the dynamic range of the image file.

Assume, for example, that there is an image file A whose bit width ofthe image file is 10 bits and whose maximum dynamic range is 100%, andthe dynamic range corresponding to given pixel values is as follows.

-   -   Image file A (maximum dynamic range: 100%)        -   Pixel value 0: dynamic range 0%        -   Pixel value 184: dynamic range 18%        -   Pixel value 1023: dynamic range 100%

In this case, conversion into a signal that represents a pixel value 0as 0 nits, a pixel value 184 as 18 nits, and a pixel value 1023 (maximumvalue) as 100 nits is performed.

Here, there are digital cameras 1 capable of changing the dynamic rangeof an image file to be recorded, by changing the settings. In suchcases, the values of the dynamic range corresponding to the pixel valuesof an image file may differ from image file to image file.

Also, assume that there is an image file B whose bit width of the imagefile is 10 bits and whose maximum dynamic range is 1000%, and thedynamic range corresponding to given pixel values is as follows.

-   -   Image file B (maximum dynamic range: 1000%)        -   Pixel value 0: dynamic range 0%        -   Pixel value 18: dynamic range 18%        -   Pixel value 1023: dynamic range 1000%

In this case, conversion into a signal that represents a pixel value 0as 0 nits, a pixel value 18 as 18 nits, and a pixel value 1023 (maximumvalue) as 1000 nits is performed.

In this way, in a digital camera capable of generating image fileshaving different dynamic ranges, generally, information relating todynamic ranges that correspond to the pixel values of image files isheld in the digital camera, for every image file format, thus enablingconversion such as described above to be performed.

By performing such conversion (hereinafter called “luminanceconversion”), given relative values will be converted into signalsrepresenting the same luminance, irrespective of whether the image filehas a dynamic range of 100% or whether the image file has a dynamicrange of 1000%, for example.

In the transmission signal generation unit 106, the received metadata1050 and the image information converted into a format suitable fortransmission are associated with each other, and transmitted to thetransmission unit 108 as a transmission signal 1060. The transmissionunit 108 is constituted by a terminal that conforms to standards such asHDMI and a control unit thereof, and transmits the transmission signal1060 sent from the transmission signal generation unit 106 to outsidethe digital camera 1.

According to the first embodiment as described above, metadata includinginformation on a common maximum luminance is appended to imageinformation obtained by converting pixel values relatively representedin the respective images into luminance values in a common dynamicrange. With the display 2, it becomes possible to handle a range ofimage data that the user wants to display at the same luminance as thesame luminance signal, by converting and displaying luminance values ofthe image information such that the common maximum luminance included inthe metadata achieves a luminance displayable by the display 2.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe second embodiment, another external digital camera is connectable tothe digital camera 1, and processing in the case of being able toreceive and hold image files from a connected external digital camerawill be described. Note that a mode in which image files are acquiredvia a portable recording medium may be adopted as a mode of acquiringimage files from another digital camera 1. Hereinafter, an example willbe described in which metadata is determined based on information on thedigital camera that generated the image file.

FIG. 3 is a block diagram representing the configuration of processingmodules in a digital camera 1 of the second embodiment, and, forexample, some or all the processing is constituted using a CPU, a memoryand the like (not shown) that are included in the digital camera 1. Theprocessing of the digital camera 1 in the second embodiment differs inthe processing of a luminance information acquisition unit 201, incomparison with the processing described with reference to FIG. 2. Thesecond embodiment is otherwise similar to the first embodiment, and thusthe same reference numbers will be given and description will be omittedas appropriate.

The luminance information acquisition unit 201, upon receiving theluminance information acquisition instruction 1020, acquires the imageinformation 1070 of the image to be displayed from the image fileholding unit 107.

FIG. 4 is a diagram showing an example of information that is includedin given image information. The image information is constituted by animage data part, which is the data of the image, and a shooting timeinformation part consisting of shooting time information. Here, shootingtime information is information on the dynamic range determined from thesettings of the camera, shooting date-time, identification informationof the camera used in shooting, and shooting mode (e.g., operation modesof the camera such as night view mode suitable for shooting nightviews), for example.

The luminance information acquisition unit 201 acquires identificationinformation of the camera used in shooting from the image information1070, and acquires dynamic range information 1040 that depends on theidentification information from the dynamic range information holdingunit 104. In the second embodiment, the dynamic range informationholding unit 104 holds dynamic range information for each cameraidentification information. Note that, in the case where there is nodynamic range information corresponding to the identificationinformation of the camera, the dynamic range information holding unit104 provides the dynamic range information of another camera instead.

The luminance information acquisition unit 201 then converts theacquired dynamic range information 1040 into luminance information 1030with a similar method to the first embodiment, and outputs the luminanceinformation 1030 to the metadata generation unit 105. In the metadatageneration unit 105, the acquired luminance information 1030 isconverted into metadata format, and output as metadata 1050.

In the transmission signal generation unit 106, the received metadata1050 and the image information converted into a format suitable fortransmission are associated with each other, and transmitted to thetransmission unit 108 as the transmission signal 1060, similarly to thefirst embodiment.

According to the second embodiment as described above, metadata thatincludes information on a common maximum luminance is appended to imageinformation obtained by converting images shot with the same camera intoluminance values in a common dynamic range. In the display 2, it therebybecomes possible to handle a range of image data that the user wants todisplay at the same luminance as the same luminance signal, with respectto images shot with the same digital camera.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe first embodiment, an example in which metadata including informationon a common maximum luminance is appended to all images in the digitalcamera 1 was described. In contrast, in the third embodiment, an examplein which images are grouped and, for every group, the same metadata isappended to images included in the group will be described.

FIG. 5 is a block diagram representing the configuration of processingmodules in a digital camera 1 of the third embodiment, and, for example,some or all the processing is constituted using a CPU, a memory and thelike (not shown) that are included in the digital camera 1. Theprocessing modules of the digital camera 1 in the third embodimentinclude a grouping processing unit 302, in comparison with theprocessing modules described with reference to FIG. 2, and, as a result,the processing of a reproduction display control unit 301, a luminanceinformation acquisition unit 303 and a transmission signal generationunit 304 differs. The third embodiment is otherwise similar to the firstembodiment, and thus the same reference numbers will be given anddescription will be omitted as appropriate.

The reproduction display control unit 301, upon receiving thereproduction control start instruction 1010, transmits a groupinginstruction 3010 to the grouping processing unit 302, transmits aluminance information acquisition instruction 3011 to the luminanceinformation acquisition unit 303, and transmits the metadata generationinstruction 1021 to the metadata generation unit 105. Also, thetransmission signal generation instruction 1022 is transmitted to thetransmission signal generation unit 304. Here, the information that istransmitted from the reproduction display control unit 301 to theprocessing modules includes information on the image to be displayed andhow the image is to be displayed.

The grouping processing unit 302 receives the grouping instruction 3010,and performs grouping of image files. The grouping referred to hereinvolves sorting image files to which the same metadata is to beappended into groups, in accordance with a grouping condition determinedin advance.

The grouping condition is, for example, that one or more of informationsuch as the shooting date-time of an image, shooting mode, setting ofpost-production image processing, shutter speed and the like are thesame or similar. The grouping condition may be preset in the digitalcamera 1, or may be selectable by the user. Also, at the time ofgrouping, grouping may be performed with consideration for the order inwhich display will be performed.

Here, an example of grouping processing will be described using FIGS. 6Ato 6C. FIG. 6A shows an example of image files that are held by theimage file holding unit 107. Note that it is assumed that display willbe performed in ascending order from the image of image file picture001,and the grouping condition is that shooting date-time and shooting modeare both the same.

FIG. 6B is an example of grouping based only on the grouping condition.In FIG. 6B, the image files picture001, picture002, picture004 andpicture005 are in group 1 and only the image file picture003 is in group2.

FIG. 6C shows an example of grouping with consideration for the order inwhich display will be performed (display order). Image files whosegrouping condition is the same are grouped in ascending order from theimage file picture001 that will be displayed first. The image files frompicture001 to picture002 meet the grouping condition, and thus form thesame group (group 1). Although the image files from picture003 aregrouped as a separate group, the next image file picture004 does notmeet the grouping condition with the image file picture003. Therefore,the image file picture003 forms a group (group 2) consisting of a singleimage file. Similarly, the remaining image files picture004 andpicture005 meet the grouping condition, and thus form the same group(group 3).

Here, grouping with consideration for the order of display is to reduceany adverse effects due to different metadata being appended if imagesincluded in a given group are not displayed continuously, since adverseeffects due to changes in image processing tend to be less perceptibleto the human eye. Also, fewer number of image files included in thegroups enables a more suitable maximum luminance to be set for theimages, thus increasing the possibility of being able to perform displayin a dynamic range suitable for the images.

The grouping processing unit 302 outputs information obtained bygrouping the image files with a method such as described above to theluminance information acquisition unit 303 and the transmission signalgeneration unit 304 as the grouping information 3020. FIG. 7 is adiagram showing an example of the grouping information 3020.

The luminance information acquisition unit 303, upon acquiring theluminance information acquisition instruction 3011 and the groupinginformation 3020, acquires luminance information for every groupdescribed in the grouping information 3020. Here, the largest dynamicrange among the dynamic ranges of the image files belonging to eachgroup is acquired, and the acquired dynamic range is replaced byluminance, which is taken as the maximum luminance information of thatgroup. For example, if the maximum dynamic range in the group is 1000%,the maximum luminance information of that group will be 1000 nits.

In the example shown in FIG. 6C, the maximum luminance information ofgroup 1 will be 200 nits, the maximum luminance information of group 2will be 300 nits, and the maximum luminance information of group 3 willbe 400 nits.

Note that, in the third embodiment, an example in which the maximumluminance information of each group is acquired from the largest dynamicrange among the dynamic ranges of the image files belonging to the groupis shown, but the method of acquiring luminance information is notlimited thereto. A table containing luminance information correspondingto each information (e.g., shooting mode) included in the shooting timeinformation of the images may be held, and the maximum luminanceinformation may be acquired from the shooting time information of theimage files of each group using the table.

The transmission signal generation unit 304 receives the transmissionsignal generation instruction 1022 and the metadata 1050, and generatesa signal for transmitting to the display 2 connected to the digitalcamera 1. Specifically, the transmission signal generation unit 304,upon receiving the transmission signal generation instruction 1022,acquires image information 1070 required in creation of the signal to betransmitted from the image file holding unit 107, and acquires thegrouping information 3020 from the grouping processing unit 302. Thetransmission signal generation unit 106 then performs luminanceconversion of the acquired image information 1070 by the methoddescribed in the first embodiment, and, furthermore, converts the imageinformation to a format suitable for signal transmission. In the thirdembodiment, the image information is converted into a format thatconforms to EOTF defined in SMPTE ST 2084 as a format suitable fortransmission. In the transmission signal generation unit 304, thereceived metadata 1050 and the image information converted into a formatsuitable for transmission are associated with each other, andtransmitted to the transmission unit 108 as a transmission signal 3040.

In this way, images are grouped, and, for every group, metadata thatincludes information on a common maximum luminance is appended to imageinformation converted into a luminance value in a common dynamic range.In the display 2, it thereby becomes possible to handle a range of imagedata that the user wants to display at the same luminance as the sameluminance signal for every group.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Inthe abovementioned first to third embodiments, the case where one pieceof metadata is generated for each image was described. In contrast, inthe fourth embodiment, an example in which common metadata that iscommon to all the images and individual metadata for each image aregenerated for each image and final metadata is generated from these twopieces of metadata depending on a slideshow reproduction interval willbe described.

Note that although the configuration of the processing modules of thedigital camera 1 in the fourth embodiment is similar to that shown inFIG. 3, the detailed contents of the processing differ, and thus theprocessing modules will be described below with reference to FIG. 3.

The user instruction acquisition unit 101 consists of a user interfacesuch as buttons, a touch panel and the like arranged on the digitalcamera 1, for example, and a processing unit thereof, and receives auser instruction 41 from a user. In the fourth embodiment, the userinstruction 41 includes a slideshow display instruction. Note that theslideshow display instruction is an instruction to perform slideshowdisplay in which image files held in the digital camera 1 are switchedat a time interval designated by the user and sequentially displayed onthe display 2 connected to the digital camera 1. This slideshow displayinstruction includes designation of the image from which to startdisplay and designation of the time interval (hereinafter, transitiontime) at which display is to be switched.

The user instruction acquisition unit 101, upon receiving the slideshowdisplay instruction from the user, transmits the reproduction controlstart instruction 1010 to the reproduction display control unit 102.

The reproduction display control unit 102, upon receiving thereproduction control start instruction 1010, transmits the luminanceinformation acquisition instruction 1020 to the luminance informationacquisition unit 201, and transmits the metadata generation instruction1021 to the metadata generation unit 105. Also, the transmission signalgeneration instruction 1022 is transmitted to the transmission signalgeneration unit 106. Here, the information that is transmitted to theprocessing modules includes information such as the image from which tostart display, the transition time of display and the like designated bythe slideshow display instruction.

The luminance information acquisition unit 201 receives the luminanceinformation acquisition instruction 1020, and acquires the maximumluminance information. The maximum luminance information in the fourthembodiment is constituted by common luminance information that is commonto all images held at the image file holding unit 107 and individualluminance information for every image. Note that, in the fourthembodiment, common luminance information that is common to all theimages held in the image file holding unit 107 is acquired, but thepresent invention is not limited thereto. For example, in the case wherethere are image files shot with other digital cameras as described inthe second embodiment, common luminance information may be acquired forevery digital camera. Also, in the case where a plurality of images aredesignated for display by the slideshow display instruction, commonluminance information that is common to the plurality of designatedimages may be acquired.

The luminance information in the fourth embodiment can be generated frominformation on the dynamic range. In view of this, the luminanceinformation acquisition unit 201, upon receiving the luminanceinformation acquisition instruction 1020, first acquires the dynamicrange information 1040 from the dynamic range information holding unit104 as common dynamic range information. Note that the common dynamicrange is, similarly to the first embodiment, a luminance range that thedigital camera 1 is capable of handling. As an example, here, assumethat the information 1000% is held in the dynamic range informationholding unit 104 as the maximum dynamic range capable of being handledby the digital camera 1.

Next, the luminance information acquisition unit 201 acquires the imageinformation 1070 from the image file holding unit 107, in order toacquire dynamic range information for every image. Image information isconstituted by an image data part, which is the data of the image, and ashooting time information part consisting of shooting time information,as described above with reference to FIG. 4. The luminance informationacquisition unit 201 sets the dynamic range information included in theshooting time information part of the acquired image information 1070 asthe dynamic range information of every image.

The luminance information acquisition unit 201 converts the acquiredcommon dynamic range information and dynamic range information for everyimage respectively into common luminance information and individualluminance information. In the fourth embodiment, similarly to the firstembodiment, the value of the dynamic range described in respectivedynamic range information is replaced by luminance, which is taken asluminance information. For example, 1000% is replaced by 1000 nits. Thecreated common luminance information and individual luminanceinformation are then combined as luminance information 1030.

The metadata generation unit 105 receives the metadata generationinstruction 1021 and the luminance information 1030, and generatesmetadata. Here, in the fourth embodiment, description is given showingdata that includes information on maximum luminance as metadata, but thetype of metadata is not limited thereto. For example, information oncolor gamut, color temperature, minimum luminance and the like may beappended as metadata. Also, for example, information on MaxCLL orMaxFALL supported by HDMI may be appended.

The metadata generation unit 105 converts the acquired luminanceinformation 1030 into metadata format (format defined as metadataformat), and outputs the converted luminance information as metadata1050. In the fourth embodiment, the metadata 1050 includes commonmetadata and individual metadata.

The transmission signal generation unit 106 receives the transmissionsignal generation instruction 1022 and the metadata 1050, and generatesa signal for transmitting to the display 2 connected to the digitalcamera 1. Specifically, the transmission signal generation unit 106,upon receiving the transmission signal generation instruction 1022,acquires image information 1070 required in creation of the signal to betransmitted from the image file holding unit 107. The transmissionsignal generation unit 106 then performs luminance conversion of theacquired image information 1070 by the method described in the firstembodiment, and, furthermore, converts the image information to a formatsuitable for signal transmission. In the fourth embodiment, the imageinformation is converted into a format that conforms to EOTF defined inSMPTE ST 2084 as a format suitable for transmission.

Next, the method of determining the maximum luminance in the fourthembodiment will be described. The transmission signal generation unit106 combines the common metadata and the individual metadata to generatemetadata for transmission on the basis of the transition time of displayof the slideshow that is included in the transmission signal generationinstruction 1022. Here, the combining ratio is such that the ratio ofcommon metadata is higher as the transition time is shorter, andconversely the ratio of individual metadata is higher as the transitiontime is longer. This is to reduce the difference in metadata betweenimages, since changes in image processing on the television side due tochanges in metadata occur more frequently as the time interval isshorter. Also, there is a strong tendency, when using individualmetadata, for image processing on the television side to be moredesirable for viewing images individually, and thus emphasis is placedon the metadata for every image in the case where the time interval islong (change in image processing does not occur often).

In view of this, in the transmission signal generation unit 106, a tablefor determining the combining ratio of common metadata and individualmetadata, according to transition time, is held in advance. Commonmetadata and individual metadata are then combined according toinformation in this table, and metadata for transmission is generated.In the transmission signal generation unit 106, the combined metadataand the image information converted into a format suitable fortransmission are then associated with each other, and transmitted to thetransmission unit 108 as the transmission signal 1060.

In this way, it becomes possible to enable more readily viewable displayto be performed, by adjusting the maximum luminance according to theslideshow transition time.

Note that whether to perform the processing shown in the abovementionedfirst to fourth embodiments can be set by a user operation, for example.In other words, a configuration may be adopted in which a mode fortransmitting, for every image, a maximum luminance that is based on thedynamic range of the image to an external device, and a mode fortransmitting the maximum luminance to an external device as in first tofourth embodiment are provided.

In the case of displaying a plurality of images, any change in luminancewhen a subject having the same luminance throughout the plurality ofimages is represented on a display device can be suppressed.

Other Embodiments

In the abovementioned first to fourth embodiments, an example in whichthe digital camera 1 generates image files that are output to thedisplay 2 was described, but the present invention is not limitedthereto. For example, the processing modules shown in FIG. 2, 3 or 5 maybe constituted in an external image processing apparatus, and imagefiles may be acquired from an image capturing apparatus. In that case,an image capturing system can be constituted by the image capturingapparatus and the image processing apparatus.

Also, the present invention may be implemented by combining theabovementioned first to fourth embodiments as appropriate.

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

1. An image processing apparatus for processing image data obtained byan image capturing apparatus, comprising one or more processors and/orcircuitry which functions as: an output unit that outputs the image dataobtained by the image capturing apparatus and maximum luminanceinformation which is information for indicating a maximum luminance ofthe image data to an external device; an acquisition unit that acquire amaximum luminance that the image capturing apparatus is capable ofhandling; and a control unit that controls the output unit to output themaximum luminance acquired by the acquisition unit as the maximumluminance information.
 2. The image processing apparatus according toclaim 1, wherein: the image capturing apparatus is capable of setting aplurality of luminance ranges, and the acquisition unit acquires themaximum luminance, among luminance included in the plurality of settableluminance ranges.
 3. The image processing apparatus according to claim1, wherein the one or more processors and/or circuitry further functionsas: a conversion unit that converts a pixel value of image data that isbased on a luminance range used in order to obtain the image data into aluminance value that is based on a luminance range that the imagecapturing apparatus is capable of handling.
 4. The image processingapparatus according to claim 3, wherein the control unit controls theoutput unit to output the luminance value obtained through conversion bythe conversion circuit.
 5. The image processing apparatus according toclaim 1, wherein: the image processing apparatus is capable ofprocessing a plurality of image data obtained by a plurality of imagecapturing apparatuses, and the acquisition unit acquires the maximumluminance that each of the plurality of image capturing apparatuses iscapable of handling, based on information of the image capturingapparatus that acquired each of the plurality of image data.
 6. Theimage processing apparatus according to claim 1, wherein the controlunit controls the output unit to output the maximum luminanceinformation as metadata of the image data.
 7. The image processingapparatus according to claim 2, wherein the luminance range is a dynamicrange.
 8. The image processing apparatus according to claim 2, whereinthe luminance range is a dynamic range.
 9. An image processing apparatuscapable of processing a plurality of image data having a plurality ofluminance ranges, comprising one or more processors and/or circuitrywhich functions as: an output unit that outputs image data to be outputand maximum luminance information which is information for indicating amaximum luminance of the image data to an external device; a sortingunit that sorts a plurality of image data obtained by image capture; anda control unit that controls the output unit to output, as the maximumluminance information, the maximum luminance of a plurality of imagedata sorted into a predetermined group, rather than the maximumluminance of the image data to be output, in a case of outputting, withthe output unit, one piece of image data sorted into the predeterminedgroup.
 10. The image processing apparatus according to claim 9, whereinthe one or more processors and/or circuitry further functions as: aconversion unit that converts a pixel value of image data that is basedon a luminance range used in the image data to be output into aluminance value that is based on a luminance range that an imagecapturing apparatus that captured the image data to be output is capableof handling.
 11. The image processing apparatus according to claim 10,wherein the control unit controls the output unit to output theluminance value obtained through conversion by the conversion unit. 12.The image processing apparatus according to claim 9, wherein t thesorting unit sorts the plurality of image data, based on informationgenerated at a time of capturing the plurality of image data.
 13. Theimage processing apparatus according to claim 9, wherein the sortingunit, in a case where images that are displayed continuously in adisplay order of the image data meet a predetermined condition, sortsthe image data of the images into the same group.
 14. The imageprocessing apparatus according to claim 9, wherein the control unitcontrols the output unit to output the maximum luminance information asmetadata of the image data.
 15. The image processing apparatus accordingto claim 9, wherein the luminance range is a dynamic range.
 16. An imageprocessing method for processing image data obtained by an imagecapturing apparatus, comprising: outputting the image data obtained bythe image capturing apparatus and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; acquiring a maximum luminance that the image capturingapparatus is capable of handling; and controlling to output the acquiredmaximum luminance as the maximum luminance information.
 17. An imageprocessing method capable of processing a plurality of image data havinga plurality of luminance ranges, comprising: outputting image data to beoutput and maximum luminance information which is information forindicating a maximum luminance of the image data to an external device;sorting a plurality of image data obtained by image capture; andcontrolling to output, as the maximum luminance information, the maximumluminance of a plurality of image data sorted into a predeterminedgroup, rather than the maximum luminance of the image data to be output,in a case of outputting, in the output step, one piece of image datasorted into the predetermined group.
 18. A non-transitorycomputer-readable storage medium storing a program for causing acomputer to function as units of the image processing apparatus forprocessing image data obtained by an image capturing apparatus,comprising: an output unit that outputs the image data obtained by theimage capturing apparatus and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; an acquisition unit that acquire a maximum luminancethat the image capturing apparatus is capable of handling; and a controlunit that controls the output unit to output the maximum luminanceacquired by the acquisition unit as the maximum luminance information.19. A non-transitory computer-readable storage medium storing a programfor causing a computer to function as units of the image processingapparatus capable of processing a plurality of image data having aplurality of luminance ranges, comprising: an output unit that outputsimage data to be output and maximum luminance information which isinformation for indicating a maximum luminance of the image data to anexternal device; a sorting unit that sorts a plurality of image dataobtained by image capture; and a control unit that controls the outputunit to output, as the maximum luminance information, the maximumluminance of a plurality of image data sorted into a predeterminedgroup, rather than the maximum luminance of the image data to be output,in a case of outputting, with the output unit, one piece of image datasorted into the predetermined group.